{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Frequently Asked Questions\n",
    "\n",
    "Here we are attempting to answer some commonly asked questions that appear on Github, and Stack Overflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import featuretools as ft\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EntitySet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I get a list of variable (column) names, and types in an `EntitySet`?\n",
    "\n",
    "After you create your `EntitySet`, you may wish to view the column names. An `EntitySet` contains multiple Dataframes, one for each entity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want view the variables (columns), and types for the \"transactions\" entity, you can do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['transactions'].variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to view the underlying Dataframe, you can do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['transactions'].df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is the difference between **copy_variables** and **additional_variables**?\n",
    "The function `normalize_entity` creates a new entity and a relationship from unique values of an existing entity. It takes 2 similar arguments:\n",
    "\n",
    "- `additional_variables` removes variables from the base entity and moves them to the new entity. \n",
    "- `copy_variables` keeps the given variables in the base entity, but also copies them to the new entity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = ft.demo.load_mock_customer()\n",
    "transactions_df = data[\"transactions\"].merge(data[\"sessions\"]).merge(data[\"customers\"])\n",
    "products_df = data[\"products\"]\n",
    "\n",
    "es = ft.EntitySet(id=\"customer_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"transactions\",\n",
    "                              dataframe=transactions_df,\n",
    "                              index=\"transaction_id\",\n",
    "                              time_index=\"transaction_time\")\n",
    "\n",
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=products_df,\n",
    "                              index=\"product_id\")\n",
    "\n",
    "new_relationship = ft.Relationship(es[\"products\"][\"product_id\"], es[\"transactions\"][\"product_id\"])\n",
    "es = es.add_relationship(new_relationship)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we normalize to create a new entity, let's look at base entity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['transactions'].df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the columns `session_id`, `session_start`, `join_date`, `device`, `customer_id`, and `zip_code`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = es.normalize_entity(base_entity_id=\"transactions\",\n",
    "                         new_entity_id=\"sessions\",\n",
    "                         index=\"session_id\",\n",
    "                         make_time_index=\"session_start\",\n",
    "                         additional_variables=[\"join_date\"],\n",
    "                         copy_variables=[\"device\", \"customer_id\", \"zip_code\",\"session_start\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we normalized the columns to create a new entity. \n",
    "- For `additional_variables`, the following column `['join_date]` will be removed from the `products` entity, and moved to the new `device` entity. \n",
    "\n",
    "\n",
    "- For `copy_variables`, the following columns `['device', 'customer_id', 'zip_code','session_start']` will be copied from the `products` entity to the new `device` entity. \n",
    "\n",
    "Let's see this in the actual `EntitySet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['transactions'].df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice above how `['device', 'customer_id', 'zip_code','session_start']` are still in the `transactions` entity, while `['join_date']` is not. But, they have all been moved to the `sessions` entity, as seen below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['sessions'].df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why did variable type change to Id, Index, or datetime_time_index?\n",
    "\n",
    "During the creation of your `EntitySet`, you might be wondering why your variable type changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = ft.demo.load_mock_customer()\n",
    "transactions_df = data[\"transactions\"].merge(data[\"sessions\"]).merge(data[\"customers\"])\n",
    "products_df = data[\"products\"]\n",
    "\n",
    "es = ft.EntitySet(id=\"customer_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"transactions\",\n",
    "                              dataframe=transactions_df,\n",
    "                              index=\"transaction_id\",\n",
    "                              time_index=\"transaction_time\")\n",
    "es.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the variable type of `session_id` is Numeric, and the variable type of `session_start` is Datetime. \n",
    "\n",
    "Now, let's normalize the transactions entity to create a new entity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = es.normalize_entity(base_entity_id=\"transactions\",\n",
    "                         new_entity_id=\"sessions\",\n",
    "                         index=\"session_id\",\n",
    "                         make_time_index=\"session_start\",\n",
    "                         additional_variables=[\"session_start\"])\n",
    "es.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The type for `session_id` is now `Id` in the `transactions` entity, and `Index` in the new entity, `sessions`. This is the case because when we normalize the entity, we create a new relationship between the `transactions` and `sessions`. There is a one to many relationship between the parent entity, `sessions`, and child entity, `transactions`.\n",
    "\n",
    "Therefore, `session_id` has type `Id` in `transactions` because it represents an `Index` in another entity. There would be a similar effect if we added another entity using `entity_from_dataframe` and `add_relationship`. \n",
    "\n",
    "In addition, when we created the new entity, we specified a `time_index` which was the variable (column) `session_start`. This changed the type of `session_start` to `datetime_time_index` in the new `sessions` entity because it now represents a time_index."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I combine two or more interesting values?\n",
    "\n",
    "You might want to create features that are conditioned on multiple values before they are calculated. This would require the use of `interesting_values`. However, since we are trying to create the feature with multiple conditions, we will need to modify the Dataframe before we create the `EntitySet`.\n",
    "\n",
    "Let's look at how you might accomplish this. \n",
    "\n",
    "First, let's create our Dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = ft.demo.load_mock_customer()\n",
    "transactions_df = data[\"transactions\"].merge(data[\"sessions\"]).merge(data[\"customers\"])\n",
    "products_df = data[\"products\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "products_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's modify our `transactions` Dataframe to create the additional column that represents multiple conditions for our feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_df['product_id_device'] = transactions_df['product_id'].astype(str) + ' and ' + transactions_df['device']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we created a new column called `product_id_device`, which just combines the `product_id` column, and the `device` column.\n",
    "\n",
    "Now let's create our `EntitySet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.EntitySet(id=\"customer_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"transactions\",\n",
    "                              dataframe=transactions_df,\n",
    "                              index=\"transaction_id\",\n",
    "                              time_index=\"transaction_time\",\n",
    "                              variable_types={\"product_id\": ft.variable_types.Categorical,\n",
    "                                              \"product_id_device\": ft.variable_types.Categorical,\n",
    "                                              \"zip_code\": ft.variable_types.ZIPCode})\n",
    "\n",
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=products_df,\n",
    "                              index=\"product_id\")\n",
    "es = es.normalize_entity(base_entity_id=\"transactions\",\n",
    "                         new_entity_id=\"sessions\",\n",
    "                         index=\"session_id\",\n",
    "                         additional_variables=[\"device\", \"product_id_device\", \"customer_id\"])\n",
    "es = es.normalize_entity(base_entity_id=\"sessions\",\n",
    "                         new_entity_id=\"customers\",\n",
    "                         index=\"customer_id\")\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we are ready to add our interesting values. \n",
    "\n",
    "First, let's view our options for what the interesting values could be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interesting_values = transactions_df['product_id_device'].unique().tolist()\n",
    "interesting_values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you wanted to, you could pick a subset of these, and the `where` features created would only use those conditions. In our example, we will use all the possible interesting values.\n",
    "\n",
    "Here, we set all of these values as our interesting values for this specific entity and variable. If we wanted to, we could make interesting values in the same way for more than one variable, but we will just stick with this one for this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['sessions']['product_id_device'].interesting_values = interesting_values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can run DFS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[\"count\"],\n",
    "                                      where_primitives=[\"count\"],\n",
    "                                      trans_primitives=[])\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To better understand the `where` clause features, let's examine one of those features. \n",
    "The feature `COUNT(sessions WHERE product_id_device = 5 and tablet)`, tells us how many sessions the customer purchased product_id 5 while on a tablet. Notice how the feature depends on multiple conditions **(product_id = 5 & device = tablet)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix[[\"COUNT(sessions WHERE product_id_device = 5 and tablet)\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Can I create an `EntitySet` using Dask dataframes? (BETA)\n",
    "\n",
    "Support for Dask EntitySets is still in Beta - if you encounter any errors using this approach, please let us know by creating a [new issue on Github](https://github.com/FeatureLabs/featuretools/issues).\n",
    "\n",
    "Yes! Featuretools supports creating an `EntitySet` from Dask dataframes. You can simply follow the same process you would when creating an `EntitySet` from pandas dataframes.\n",
    "\n",
    "There are some limitations to be aware of when using Dask dataframes. When creating an `Entity` from a Dask dataframe, variable type inference is not performed as it is for pandas entities, so the user must supply a list of variable types during creation. Also, other quality checks are not performed, such as checking for unique index values. An `EntitySet` must be created entirely of Dask entities or pandas entities - you cannot mix pandas entities with Dask entitites in the same `EntitySet`.\n",
    "\n",
    "For more information on creating an `EntitySet` from Dask dataframes, see the [Using Dask EntitySets](https://docs.featuretools.com/en/stable/guides/using_dask_entitysets.html) guide."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DFS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why is DFS not creating aggregation features?\n",
    "You may have created your `EntitySet`, and then applied DFS to create features. However, you may be puzzled as to why no aggregation features were created. \n",
    "\n",
    "- **This is most likely because you have a single table in your entity, and DFS is not capable of creating aggregation features with fewer than 2 entities. Featuretools looks for a relationship, and aggregates based on that relationship.**\n",
    "\n",
    "Let's look at a simple example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = ft.demo.load_mock_customer()\n",
    "transactions_df = data[\"transactions\"].merge(data[\"sessions\"]).merge(data[\"customers\"])\n",
    "\n",
    "es = ft.EntitySet(id=\"customer_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"transactions\",\n",
    "                              dataframe=transactions_df,\n",
    "                              index=\"transaction_id\")\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we only have 1 entity in our `EntitySet`. If we try to create aggregation features on this `EntitySet`, it will not be possible because DFS needs 2 entities to generate aggregation features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es, target_entity=\"transactions\")\n",
    "feature_defs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "None of the above features are aggregation features. To fix this issue, you can add another entity to your `EntitySet`.\n",
    "\n",
    "**Solution #1 - You can add new entity if you have additional data.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "products_df = data[\"products\"]\n",
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=products_df,\n",
    "                              index=\"product_id\")\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we now have an additional entity in our `EntitySet`, called `products`.\n",
    "\n",
    "**Solution #2 - You can normalize an existing entity.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = es.normalize_entity(base_entity_id=\"transactions\",\n",
    "                         new_entity_id=\"sessions\",\n",
    "                         index=\"session_id\",\n",
    "                         make_time_index=\"session_start\",\n",
    "                         additional_variables=[\"device\", \"customer_id\", \"zip_code\", \"join_date\"],\n",
    "                         copy_variables=[\"session_start\"])\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we now have an additional entity in our `EntitySet`, called `sessions`. Here, the normalization created a relationship between `transactions` and `sessions`. However, we could have specified a relationship between `transactions` and `products` if we had only used Solution \\#1.\n",
    "\n",
    "Now, we can generate aggregation features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es, target_entity=\"transactions\")\n",
    "feature_defs[:-10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few of the aggregation features are:\n",
    "\n",
    "- `<Feature: sessions.SUM(transactions.amount)>`\n",
    "- `<Feature: sessions.STD(transactions.amount)>`\n",
    "- `<Feature: sessions.MAX(transactions.amount)>`\n",
    "- `<Feature: sessions.SKEW(transactions.amount)>`\n",
    "- `<Feature: sessions.MIN(transactions.amount)>`\n",
    "- `<Feature: sessions.MEAN(transactions.amount)>`\n",
    "- `<Feature: sessions.COUNT(transactions)>`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I speed up the runtime of DFS?\n",
    "\n",
    "One issue you may encounter while running `ft.dfs` is slow performance. While Featuretools has generally optimal default settings for calculating features, you may want to speed up performance when you are calculating on a large number of features. \n",
    "\n",
    "One quick way to speed up performance is by adjusting the `n_jobs` settings of `ft.dfs` or `ft.calculate_feature_matrix`.\n",
    "\n",
    "```python\n",
    "# setting n_jobs to -1 will use all cores\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      n_jobs=-1)\n",
    "\n",
    "                                      \n",
    "feature_matrix, feature_defs = ft.calculate_feature_matrix(entityset=es,\n",
    "                                                           features=feature_defs,\n",
    "                                                           n_jobs=-1)\n",
    "```\n",
    "\n",
    "\n",
    "**For more ways to speed up performance, please visit:**\n",
    "\n",
    "- [Improving Computational Performance](./guides/performance.rst#improving-computational-performance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I include only certain features when running DFS?\n",
    "\n",
    "When using DFS to generate features, you may wish to include only certain features. There are multiple ways that you do this:\n",
    "\n",
    "- Use the `ignore_variables` to specify variables in an entity that should not be used to create features. It is a dictionary mapping an entity id to a list of variable names to ignore.\n",
    "\n",
    "- Use `drop_contains` to drop features that contain any of the strings listed in this parameter.\n",
    "\n",
    "- Use `drop_exact` to drop features that exactly match any of the strings listed in this parameter.\n",
    "\n",
    "Here is an example of using all three parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      ignore_variables={\n",
    "                                           \"transactions\": [\"amount\"],\n",
    "                                           \"customers\": [\"age\", \"gender\", \"date_of_birth\"]\n",
    "                                       }, # ignore these variables\n",
    "                                      drop_contains=[\"customers.SUM(\"], # drop features that contain these strings\n",
    "                                      drop_exact=[\"STD(transactions.quanity)\"]) # drop features that exactly match"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I specify primitives on a per column or per entity basis?\n",
    "\n",
    "When using DFS to generate features, you may wish to use only certain features or entities for specific primitives. This can be done through the `primitive_options` parameter. The `primitive_options` parameter is a dictionary that maps a primitive or a tuple of primitives to a dictionary containing options for the primitive(s). A primitive or tuple of primitives can also be mapped to a list of option dictionaries if the primitive(s) \n",
    "takes multiple inputs. The primitive keys can be the string names of the primitive, the primitive class, or specific instances of the primitive. Each dictionary supplies options for their respective input column. There are multiple ways to control how primitives get applied through these options:\n",
    "\n",
    "- Use `ignore_entities` to specify entities that should not be used to create features for that primitive. It is a list of entity ids to ignore.\n",
    "\n",
    "- Use `include_entities` to specify the only entities to be included to create features for that primitive. It is a list of entity ids to include.\n",
    "\n",
    "- Use `ignore_variables` to specify variables in an entity that should not be used to create features for that primitive. It is a dictionary mapping an entity id to a list of variable names to ignore.\n",
    "\n",
    "- Use `include_variables` to specify the only variables in an entity that should be used to create features for that primitive. It is a dictionary mapping an entity id to a list of variable names to include.\n",
    "\n",
    "You can also use `primitive_options` to specify which entities or variables you wish to use as groupbys for groupby transformation primitives:\n",
    "\n",
    "- Use `ignore_groupby_entities` to specify entities that should not be used to get groupbys for that primitive. It is a list of entity ids to ignore.\n",
    "\n",
    "- Use `include_groupby_entities` to specify the only entities that should be used to get groupbys for that primitive. It is a list of entity ids to include.\n",
    "\n",
    "- Use `ignore_groupby_variables` to specify variables in an entity that should not be used as groupbys for that primitive. It is a dictionary mapping an entity id to a list of variable names to ignore.\n",
    "\n",
    "- Use `include_groupby_variables` to specify the only variables in an entity that should be used as groupbys for that primitive. It is a dictionary mapping an entity id to a list of variable names to include.\n",
    "\n",
    "Here is an example of using some of these options:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      primitive_options={\"mode\": {\"ignore_entities\": [\"sessions\"],\n",
    "                                                                  \"include_variables\": {\"products\": [\"brand\"],\n",
    "                                                                                        \"transactions\": [\"product_id\"]}},\n",
    "                                                         # For mode, ignore the \"sessions\" entity and only include \"brands\" in the\n",
    "                                                         # \"products\" entity and \"product_id\" in the \"transactions\" entity\n",
    "                                                         (\"count\", \"mean\"): {\"include_entities\": [\"sessions\", \"transactions\"]}\n",
    "                                                         # For count and mean, only include the entities \"sessions\" and \"transactions\"\n",
    "                                                         })\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if options are given for a specific instance of a primitive and for the primitive generally (either by string name or class), the instances with their own options will not use the generic options. For example, in this case:\n",
    "```\n",
    "special_mean = Mean()\n",
    "options = {\n",
    "    special_mean: {'include_entities: ['customers']},\n",
    "    'mean': {'include_entities: ['sessions']}\n",
    "```\n",
    "the primitive `special_mean` will not use the entity `sessions` because it's options have it only include `customers`. Every other instance of the `Mean` primitive will use the `'mean'` options.  \n",
    "\n",
    "**For a more examples of specifying options for DFS, please visit:**\n",
    "\n",
    "- [Specifying Primitive Options](guides/specifying_primitive_options.rst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### If I didn't specify the **cutoff_time**, what date will be used for the feature calculations?\n",
    "\n",
    "The cutoff time will be set to the current time using `cutoff_time = datetime.now()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I select a certain amount of past data when calculating features?\n",
    "\n",
    "You may encounter a situation when you wish to make prediction using only a certain amount of historical data. You can accomplish this using the `training_window` parameter in `ft.dfs`. When you use the `training_window`, Featuretools will use the historical data between the `cutoff_time` and `cutoff_time - training_window`.\n",
    "\n",
    "In order to make the calculation, Featuretools will check the time in the `time_index` column of the `target_entity`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "es['customers'].time_index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our target_entity has a `time_index`, which is needed for the `training_window` calculation. Here, we are creating a cutoff time dataframe so that we can have a unique training window for each customer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cutoff_times = pd.DataFrame()\n",
    "cutoff_times['customer_id'] = [1, 2, 3, 1]\n",
    "cutoff_times['time'] = pd.to_datetime(['2014-1-1 04:00', '2014-1-1 05:00', '2014-1-1 06:00', '2014-1-1 08:00'])\n",
    "cutoff_times['label'] = [True, True, False, True]\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      cutoff_time=cutoff_times,\n",
    "                                      cutoff_time_in_index=True,\n",
    "                                      training_window=\"1 hour\")\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we ran DFS with `training_window` argument of `1 hour` to create features that only used customer data collected in the last hour (from the cutoff time we provided)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I apply DFS to a single table?\n",
    "\n",
    "You can run DFS on a single table. Featuretools will be able to generate features for your data, but only transform features.\n",
    "\n",
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_df = ft.demo.load_mock_customer(return_single_table=True)\n",
    "\n",
    "es = ft.EntitySet(id=\"customer_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"transactions\",\n",
    "                              dataframe=transactions_df,\n",
    "                              index=\"transaction_id\",\n",
    "                              time_index=\"transaction_time\")\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es, \n",
    "                                      target_entity=\"transactions\", \n",
    "                                      trans_primitives=['time_since', 'day', 'is_weekend', \n",
    "                                                        'cum_min', 'minute', \n",
    "                                                        'num_words', 'weekday', 'cum_count', \n",
    "                                                        'percentile', 'year', 'week',\n",
    "                                                        'cum_mean'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we examine the output, let's look at our original single table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can look at the transformations that Featuretools was able to apply to this single entity (table) to create feature matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Can I automatically normalize a single table?\n",
    "\n",
    "Yes, another open source library [AutoNormalize](https://github.com/FeatureLabs/autonormalize), also produced by Feature Labs, automates table normalization and integrates with Featuretools. To install run:\n",
    "\n",
    "```shell\n",
    "python -m pip install featuretools[autonormalize]\n",
    "```\n",
    "\n",
    "A normalized `EntitySet` will help Featuretools to generate more features. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.autonormalize import autonormalize as an\n",
    "es = an.normalize_entity(es)\n",
    "es.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, AutoNormalize creates a relational `EntitySet`. Below, we run dfs on the `EntitySet`, and you can see all the features created; take note of the aggregated features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es, \n",
    "                                      target_entity=\"transaction_id\", \n",
    "                                      trans_primitives=[])\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I prevent label leakage with DFS?\n",
    "\n",
    "One concern you might have with using DFS is about label leakage. You want to make sure that labels in your data aren't used incorrectly to create features and the feature matrix.\n",
    "\n",
    "**Featuretools is particularly focused on helping users avoid label leakage.**\n",
    "\n",
    "There are two ways to prevent label leakage depending on if your data has timestamps or not.\n",
    "\n",
    "#### 1. Data without timestamps\n",
    "In the case where you do not have timestamps, you can create one `EntitySet` using only the training data and then run `ft.dfs`. This will create a feature matrix using only the training data, but also return a list of feature definitions. Next, you can create an `EntitySet` using the test data and recalculate the same features by calling `ft.calculate_feature_matrix` with the list of feature definitions from before. \n",
    "\n",
    "Here is what that flow would look like:\n",
    "\n",
    "First, let's create our training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                           \"age\": [40, 50, 10, 20, 30],\n",
    "                           \"gender\": [\"m\", \"f\", \"m\", \"f\", \"f\"],\n",
    "                           \"signup_date\": pd.date_range('2014-01-01 01:41:50', periods=5, freq='25min'),\n",
    "                           \"labels\": [True, False, True, False, True]})\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can create an entityset for our training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es_train_data = ft.EntitySet(id=\"customer_train_data\")\n",
    "es_train_data = es_train_data.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                                    dataframe=train_data,\n",
    "                                                    index=\"customer_id\")\n",
    "es_train_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we are ready to create our features, and feature matrix for the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix_train, feature_defs = ft.dfs(entityset=es_train_data, \n",
    "                                            target_entity=\"customers\")\n",
    "feature_matrix_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also encode our feature matrix to make machine learning compatible features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix_train_enc, features_enc = ft.encode_features(feature_matrix_train, feature_defs)\n",
    "feature_matrix_train_enc.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the the whole feature matrix only inclues numeric values now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use the feature definitions to calculate our feature matrix for the test data, and avoid label leakage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_train = pd.DataFrame({\"customer_id\": [6, 7, 8, 9, 10],\n",
    "                           \"age\": [20, 25, 55, 22, 35],\n",
    "                           \"gender\": [\"f\", \"m\", \"m\", \"m\", \"m\"],\n",
    "                           \"signup_date\": pd.date_range('2014-01-01 01:41:50', periods=5, freq='25min')})\n",
    "\n",
    "# lets add NaN label column to the test Dataframe\n",
    "test_train['labels'] = np.nan\n",
    "\n",
    "es_test_data = ft.EntitySet(id=\"customer_test_data\")\n",
    "es_test_data = es_test_data.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                                  dataframe=test_train,\n",
    "                                                  index=\"customer_id\",\n",
    "                                                  time_index=\"signup_date\")\n",
    "\n",
    "# Use the feature definitions from earlier\n",
    "feature_matrix_enc_test = ft.calculate_feature_matrix(features=features_enc, \n",
    "                                                      entityset=es_test_data)\n",
    "\n",
    "feature_matrix_enc_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note: Disregard the difference between the False/True above, and 0/1 in the earlier feature matrix. A simple casting would address this difference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Data with timestamps\n",
    "\n",
    "If your data has timestamps, the best way to prevent label leakage is to use a list of **cutoff times**, which specify the last point in time data is allowed to be used for each row in the resulting feature matrix. To use **cutoff times**, you need to set a time index for each time sensitive entity in your entity set.\n",
    "\n",
    "> **Tip: Even if your data doesn’t have time stamps, you could add a column with dummy timestamps that can be used by Featuretools as time index.**\n",
    "\n",
    "When you call `ft.dfs`, you can provide a Dataframe of cutoff times like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cutoff_times = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                             \"time\": pd.date_range('2014-01-01 01:41:50', periods=5, freq='25min')})\n",
    "cutoff_times.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_test_data = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                                \"age\": [20, 25, 55, 22, 35],\n",
    "                                \"gender\": [\"f\", \"m\", \"m\", \"m\", \"m\"],\n",
    "                                \"signup_date\": pd.date_range('2010-01-01 01:41:50', periods=5, freq='25min')})\n",
    "\n",
    "es_train_test_data = ft.EntitySet(id=\"customer_train_test_data\")\n",
    "es_train_test_data = es_train_test_data.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                                              dataframe=train_test_data,\n",
    "                                                              index=\"customer_id\",\n",
    "                                                              time_index=\"signup_date\")\n",
    "\n",
    "feature_matrix_train_test, features = ft.dfs(entityset=es_train_test_data,\n",
    "                                             target_entity=\"customers\",\n",
    "                                             cutoff_time=cutoff_times,\n",
    "                                             cutoff_time_in_index=True)\n",
    "feature_matrix_train_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we have created a feature matrix that uses cutoff times to avoid label leakage. We could also encode this feature matrix using `ft.encode_features`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is the difference between passing a primitive object versus a string to DFS?  \n",
    "\n",
    "There are 2 ways to pass primitives to DFS: the primitive object, or a string of the primitive name. \n",
    "\n",
    "We will use the Transform primitive called `TimeSincePrevious` to illustrate the differences.\n",
    "\n",
    "First, let's use the string of primitive name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[],\n",
    "                                      trans_primitives=[\"time_since_previous\"])\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's use the primitive object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import TimeSincePrevious\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[],\n",
    "                                      trans_primitives=[TimeSincePrevious])\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see above, the feature matrix is the same.\n",
    "\n",
    "However, if we need to modify controllable parameters in the primitive, we should use the primitive object. \n",
    "For instance, let's make TimeSincePrevious return units of hours (the default is in seconds)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import TimeSincePrevious\n",
    "\n",
    "time_since_previous_in_hours = TimeSincePrevious(unit='hours')\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[],\n",
    "                                      trans_primitives=[time_since_previous_in_hours])\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How can I select features based on some attributes (a specific string, an explicit primitive type, a return type, a given depth)?\n",
    "\n",
    "You may wish to select a subset of your features based on some attributes. \n",
    "\n",
    "Let's say you wanted to select features that had the string `amount` in its name. You can check for this by using the `get_name` function on the feature definitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "\n",
    "feature_defs = ft.dfs(entityset=es,\n",
    "                      target_entity=\"customers\",\n",
    "                      features_only=True)\n",
    "\n",
    "features_with_amount = []\n",
    "for x in feature_defs:\n",
    "    if 'amount' in x.get_name():\n",
    "        features_with_amount.append(x)\n",
    "features_with_amount[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You might also want to only select features that are aggregation features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools import AggregationFeature\n",
    "\n",
    "features_only_aggregations = []\n",
    "for x in feature_defs:\n",
    "    if type(x) == AggregationFeature:\n",
    "        features_only_aggregations.append(x)\n",
    "features_only_aggregations[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, you might only want to select features that are calculated at a certain depth. You can do this by using the `get_depth` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_only_depth_2 = []\n",
    "for x in feature_defs:\n",
    "    if x.get_depth() == 2:\n",
    "        features_only_depth_2.append(x)\n",
    "features_only_depth_2[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, you might only want features that return a certain type. You can do this by using the `variable_type` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.variable_types import Numeric\n",
    "\n",
    "features_only_numeric = []\n",
    "for x in feature_defs:\n",
    "    if x.variable_type == Numeric:\n",
    "        features_only_numeric.append(x)\n",
    "features_only_numeric[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you have your specific feature list, you can use `ft.calculate_feature_matrix` to generate a feature matrix for only those features.\n",
    "\n",
    "For our example, let's use the features with only the string `amount` in its name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix = ft.calculate_feature_matrix(entityset=es,\n",
    "                                             features=features_with_amount) # change to your specific feature list\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, notice how all the column names for our feature matrix contain the string `amount`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I create **where** features?\n",
    "\n",
    "Sometimes, you might want to create features that are conditioned on a second value before it is calculated. This extra filter is called a “where clause”. You can create these features using the using the `interesting_values` of a variable.\n",
    "\n",
    "If you have categorical columns in your `EntitySet`, you can use then `add_interesting_values`. This function will  find interesting values for your categorical variables, which can then be used to generate “where” clauses.\n",
    "\n",
    "First, let's create our `EntitySet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can add the interesting variables for the categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es.add_interesting_values()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can run DFS with the `where_primitives` argument to define which primitives to apply with where clauses. In this case, let's use the primitive `count`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[\"count\"],\n",
    "                                      where_primitives=[\"count\"],\n",
    "                                      trans_primitives=[])\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now created some useful features. One example of a useful feature is the `COUNT(sessions WHERE device = tablet)`. This feature tells us how many sessions a customer completed on a tablet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix[[\"COUNT(sessions WHERE device = tablet)\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Primitives"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is the difference between the primitive types (Transform, GroupBy Transform, & Aggregation)?\n",
    "\n",
    "You might curious to know the difference between the primitive groups.\n",
    "Let's review the differences between transform, groupby transform, and aggregation primitives.\n",
    "\n",
    "First, let's create a simple `EntitySet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import featuretools as ft\n",
    "\n",
    "df = pd.DataFrame({\n",
    "    \"id\": [1, 2, 3, 4, 5, 6],\n",
    "    \"time_index\": pd.date_range(\"1/1/2019\", periods=6, freq=\"D\"),\n",
    "    \"group\": [\"a\", \"b\", \"a\", \"c\", \"a\", \"b\"],\n",
    "    \"val\": [5, 1, 10, 20, 6, 23],\n",
    "})\n",
    "es = ft.EntitySet()\n",
    "es = es.entity_from_dataframe(entity_id=\"observations\",\n",
    "                              dataframe=df,\n",
    "                              index=\"id\",\n",
    "                              time_index=\"time_index\")\n",
    "\n",
    "es = es.normalize_entity(base_entity_id=\"observations\",\n",
    "                         new_entity_id=\"groups\",\n",
    "                         index=\"group\")\n",
    "\n",
    "es.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After calling normalize_entity, the variable \"group\" has the type \"id\" because it identifies another entity. Alternatively, it could be set using the variable_types parameter when we first call `es.entity_from_dataframe()`.\n",
    "\n",
    "#### Transform Primitive\n",
    "\n",
    "The cum_sum primitive calculates the running sum in list of numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import CumSum\n",
    "\n",
    "cum_sum = CumSum()\n",
    "cum_sum([1, 2, 3, 4, 5]).tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we apply it using the trans_primitives argument it will calculate it over the entire observations entity like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(target_entity=\"observations\",\n",
    "                                      entityset=es,\n",
    "                                      agg_primitives=[],\n",
    "                                      trans_primitives=[\"cum_sum\"],\n",
    "                                      groupby_trans_primitives=[])\n",
    "\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Groupby Transform Primitive\n",
    "\n",
    "If we apply it using `groupby_trans_primitives`, then DFS will first group by any id variables before applying the transform primitive. As a result, we get the cumulative sum by group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(target_entity=\"observations\",\n",
    "                                      entityset=es,\n",
    "                                      agg_primitives=[],\n",
    "                                      trans_primitives=[],\n",
    "                                      groupby_trans_primitives=[\"cum_sum\"])\n",
    "\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Aggregation Primitive\n",
    "\n",
    "Finally, there is also the aggregation primitive \"sum\". If we use sum, it will calculate the sum for the group at the cutoff time for each row. Because we didn't specify a cutoff time it will use all the data for each group for each row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(target_entity=\"observations\",\n",
    "                                      entityset=es,\n",
    "                                      agg_primitives=[\"sum\"],\n",
    "                                      trans_primitives=[],\n",
    "                                      cutoff_time_in_index=True,\n",
    "                                      groupby_trans_primitives=[])\n",
    "\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we set the cutoff time of each row to be the time index, then use sum as an aggregation primitive, the result is the same as cum_sum. (Though the order is different in the displayed dataframe)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cutoff_time = df[[\"id\", \"time_index\"]]\n",
    "cutoff_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, feature_defs = ft.dfs(target_entity=\"observations\",\n",
    "                                      entityset=es,\n",
    "                                      agg_primitives=[\"sum\"],\n",
    "                                      trans_primitives=[],\n",
    "                                      groupby_trans_primitives=[],\n",
    "                                      cutoff_time_in_index=True,\n",
    "                                      cutoff_time=cutoff_time)\n",
    "\n",
    "feature_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I get a list of all Aggregation and Transform primitives?\n",
    "\n",
    "You can do `featuretools.list_primitives()` to get all the primitive in Featuretools. It will return a Dataframe with the names, type, and description of the primitives, and if the primitive can be used with entitysets created from Dask dataframes. You can also visit [primitives.featurelabs.com](https://primitives.featurelabs.com/) to obtain a list of all available primitives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_primitives = ft.list_primitives()\n",
    "df_primitives.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_primitives.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What primitives can I use when creating a feature matrix from a Dask `EntitySet`? (BETA)\n",
    "\n",
    "Support for Dask EntitySets is still in Beta - if you encounter any errors using this approach, please let us know by creating a [new issue on Github](https://github.com/FeatureLabs/featuretools/issues).\n",
    "\n",
    "When creating a feature matrix from a Dask `EntitySet`, only certain primitives can be used. Computation of certain features is quite expensive in a distributed environment, and as a result only a subset of Featuretools primitives are currently supported when using a Dask `EntitySet`.\n",
    "\n",
    "The table returned by `featuretools.list_primitives()` will contain a column labeled `dask_compatible`. Any primitive that has a value of `True` in this column can be used safely when computing a feature matrix from a Dask `EntitySet`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How do I change the units for a TimeSince primitive?\n",
    "There are a few primitives in Featuretools that make some time-based calculation. These include `TimeSince, TimeSincePrevious, TimeSinceLast, TimeSinceFirst`. \n",
    "\n",
    "You can change the units from the default seconds to any valid time unit, by doing the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import TimeSince, TimeSincePrevious, TimeSinceLast, TimeSinceFirst\n",
    "\n",
    "time_since = TimeSince(unit=\"minutes\")\n",
    "time_since_previous = TimeSincePrevious(unit=\"hours\")\n",
    "time_since_last = TimeSinceLast(unit=\"days\")\n",
    "time_since_first = TimeSinceFirst(unit=\"years\")\n",
    "\n",
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      agg_primitives=[time_since_last, time_since_first],\n",
    "                                      trans_primitives=[time_since, time_since_previous])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we changed the units to the following:\n",
    "- minutes for `TimeSince`\n",
    "- hours for `TimeSincePrevious`\n",
    "- days for `TimeSinceLast`\n",
    "- years for `TimeSinceFirst`.\n",
    "\n",
    "\n",
    "Now we can see that our feature matrix contains multiple features where the units for the TimeSince primitives are changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are now features where time unit is different from the default of seconds, such as `TIME_SINCE_LAST(sessions.session_start, unit=days)`, and `TIME_SINCE_FIRST(sessions.session_start, unit=years)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How does my train & test data work with Featuretools and sklearn's **train_test_split**?\n",
    "\n",
    "You might be wondering how to properly use your train & test data with Featuretools, and sklearn's **train_test_split**. There are a few things you must do to ensure accuracy with this workflow.\n",
    "\n",
    "Let's imagine we have a Dataframes for our train data, with the labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                           \"age\": [20, 25, 55, 22, 35],\n",
    "                           \"gender\": [\"f\", \"m\", \"m\", \"m\", \"m\"],\n",
    "                           \"signup_date\": pd.date_range('2010-01-01 01:41:50', periods=5, freq='25min'),\n",
    "                           \"labels\": [False, True, True, False, False]})\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create our `EntitySet` for the train data, and create our features. To prevent label leakage, we will use cutoff times (see [earlier question](#How-do-I-prevent-label-leakage-with-DFS?))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es_train_data = ft.EntitySet(id=\"customer_data\")\n",
    "es_train_data = es_train_data.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                                    dataframe=train_data,\n",
    "                                                    index=\"customer_id\")\n",
    "\n",
    "cutoff_times = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                             \"time\": pd.date_range('2014-01-01 01:41:50', periods=5, freq='25min')})\n",
    "\n",
    "feature_matrix_train, features = ft.dfs(entityset=es_train_data,\n",
    "                                        target_entity=\"customers\",\n",
    "                                        cutoff_time=cutoff_times,\n",
    "                                        cutoff_time_in_index=True)\n",
    "feature_matrix_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also encode our feature matrix to compatible for machine learning algorithms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix_train_enc, feature_enc = ft.encode_features(feature_matrix_train, features)\n",
    "feature_matrix_train_enc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X = feature_matrix_train_enc.drop(['labels'], axis=1)\n",
    "y = feature_matrix_train_enc['labels']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can use the encoded feature matrix with sklearn's **train_test_split**. This will allow you to train your model, and tune your parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How are categorical variables encoded when splitting training and testing data?\n",
    "\n",
    "You might be wondering what happens when categorical variables are encoded with your training and testing data. You might be curious to know what happens if the train data has a categorical variable that is not present in the testing data. \n",
    "\n",
    "Let's explore a simple example to see what happens during the encoding process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = pd.DataFrame({\"customer_id\": [1, 2, 3, 4, 5],\n",
    "                           \"product_purchased\": [\"coke zero\", \"car\", \"toothpaste\", \"coke zero\", \"car\"]})\n",
    "\n",
    "es_train = ft.EntitySet(id=\"customer_data\")\n",
    "es_train = es_train.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                          dataframe=train_data,\n",
    "                                          index=\"customer_id\")\n",
    "\n",
    "feature_matrix_train, features = ft.dfs(entityset=es_train,\n",
    "                                        target_entity='customers')\n",
    "feature_matrix_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use `ft.encode_features` to properly encode the product_purchased column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix_train_encoded, features_encoded = ft.encode_features(feature_matrix_train,\n",
    "                                                                    features)\n",
    "feature_matrix_train_encoded.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets imagine we have some test data that has doesn't have one of the categorical values (**toothpaste**). Also, the test data has a value that wasn't present in the train data (**water**)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = pd.DataFrame({\"customer_id\": [6, 7, 8, 9, 10],\n",
    "                          \"product_purchased\": [\"coke zero\", \"car\", \"coke zero\", \"coke zero\", \"water\"]})\n",
    "\n",
    "es_test = ft.EntitySet(id=\"customer_data\")\n",
    "es_test = es_test.entity_from_dataframe(entity_id=\"customers\",\n",
    "                                        dataframe=test_data,\n",
    "                                        index=\"customer_id\")\n",
    "\n",
    "feature_matrix_test = ft.calculate_feature_matrix(entityset=es_test,\n",
    "                                                  features=features_encoded)\n",
    "feature_matrix_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen above, we were able to successfully handle the encoding, and deal with the following complications: \n",
    "- **toothpaste** was present in the training data but not present in the testing data \n",
    "- **water** was present in the test data but not present in the training data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Errors & Warnings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why am I getting this error 'Index is not unique on dataframe'?\n",
    "You may be trying to create your `EntitySet`, and run into this error. \n",
    "```python\n",
    "AssertionError: Index is not unique on dataframe\n",
    "```\n",
    "**This is because each entity in your EntitySet needs a unique index.**\n",
    "\n",
    "Let's look at a simple example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_df = pd.DataFrame({'id': [1, 2, 3, 4, 4],\n",
    "                           'rating': [3.5, 4.0, 4.5, 1.5, 5.0]})\n",
    "product_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the `id` column has a duplicate index of `4`. If you try to create an entity with this Dataframe, you will run into the following error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "es = ft.EntitySet(id=\"product_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=product_df,\n",
    "                              index=\"id\")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "---------------------------------------------------------------------------\n",
    "AssertionError                            Traceback (most recent call last)\n",
    "<ipython-input-63-a6e02ba6fa47> in <module>\n",
    "      2 es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "      3                               dataframe=product_df,\n",
    "----> 4                               index=\"id\")\n",
    "\n",
    "~/featuretools/featuretools/entityset/entityset.py in entity_from_dataframe(self, entity_id, dataframe, index, variable_types, make_index, time_index, secondary_time_index, already_sorted)\n",
    "    486             secondary_time_index=secondary_time_index,\n",
    "    487             already_sorted=already_sorted,\n",
    "--> 488             make_index=make_index)\n",
    "    489         self.entity_dict[entity.id] = entity\n",
    "    490         self.reset_data_description()\n",
    "\n",
    "~/featuretools/featuretools/entityset/entity.py in __init__(self, id, df, entityset, variable_types, index, time_index, secondary_time_index, last_time_index, already_sorted, make_index, verbose)\n",
    "     79 \n",
    "     80         self.df = df[[v.id for v in self.variables]]\n",
    "---> 81         self.set_index(index)\n",
    "     82 \n",
    "     83         self.time_index = None\n",
    "\n",
    "~/featuretools/featuretools/entityset/entity.py in set_index(self, variable_id, unique)\n",
    "    450         self.df.index.name = None\n",
    "    451         if unique:\n",
    "--> 452             assert self.df.index.is_unique, \"Index is not unique on dataframe (Entity {})\".format(self.id)\n",
    "    453 \n",
    "    454         self.convert_variable_type(variable_id, vtypes.Index, convert_data=False)\n",
    "\n",
    "AssertionError: Index is not unique on dataframe (Entity products)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To fix the above error, you can do one of the following solutions:\n",
    "\n",
    "**Solution #1 - You can create a unique index on your Dataframe.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_df = pd.DataFrame({'id': [1, 2, 3, 4, 5],\n",
    "                           'rating': [3.5, 4.0, 4.5, 1.5, 5.0]})\n",
    "product_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we now have a unique index column called `id`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=product_df,\n",
    "                              index=\"id\")\n",
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen above, we can now create our entity for our `EntitySet` without an error by creating a unique index in our Dataframe.\n",
    "\n",
    "**Solution #2 - Set make_index to True in your call to entity_from_dataframe to create a new index on that data**\n",
    "- `make_index` creates a unique index for each row by just looking at what number the row is, in relation to all the other rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "product_df = pd.DataFrame({'id': [1, 2, 3, 4, 4],\n",
    "                           'rating': [3.5, 4.0, 4.5, 1.5, 5.0]})\n",
    "\n",
    "es = ft.EntitySet(id=\"product_data\")\n",
    "es = es.entity_from_dataframe(entity_id=\"products\",\n",
    "                              dataframe=product_df,\n",
    "                              index=\"product_id\",\n",
    "                              make_index=True)\n",
    "es['products'].df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen above, we created our entity for our `EntitySet` without an error using the `make_index` argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why am I getting the following warning 'Using training_window but last_time_index is not set'?\n",
    "\n",
    "If you are using a training window, and you haven't set a `last_time_index` for your entity, you will get this warning.\n",
    "The training window attribute in Featuretools limits the amount of past data that can be used while calculating a particular feature vector.\n",
    "\n",
    "You can add the `last_time_index` to all entities automatically by calling `your_entityset.add_last_time_indexes()` after you create your `EntitySet`. This will remove the warning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.demo.load_mock_customer(return_entityset=True)\n",
    "es.add_last_time_indexes()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can run DFS without getting the warning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cutoff_times = pd.DataFrame()\n",
    "cutoff_times['customer_id'] = [1, 2, 3, 1]\n",
    "cutoff_times['time'] = pd.to_datetime(['2014-1-1 04:00', '2014-1-1 05:00', '2014-1-1 06:00', '2014-1-1 08:00'])\n",
    "cutoff_times['label'] = [True, True, False, True]\n",
    "\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es,\n",
    "                                      target_entity=\"customers\",\n",
    "                                      cutoff_time=cutoff_times,\n",
    "                                      cutoff_time_in_index=True,\n",
    "                                      training_window=\"1 hour\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### last_time_index vs. time_index\n",
    "\n",
    "- The `time_index` is when the instance was first known.\n",
    "- The `last_time_index` is when the instance appears for the last time.\n",
    "- For example, a customer’s session has multiple transactions which can happen at different points in time. If we are trying to count the number of sessions a user has in a given time period, we often want to count all the sessions that had any transaction during the training window. To accomplish this, we need to not only know when a session starts (**time_index**), but also when it ends (**last_time_index**). The last time that an instance appears in the data is stored as the last_time_index of an Entity. \n",
    "- Once the last_time_index has been set, Featuretools will check to see if the last_time_index is after the start of the training window. That, combined with the cutoff time, allows DFS to discover which data is relevant for a given training window."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why am I getting errors with Featuretools on [Google Colab](https://colab.research.google.com/)?\n",
    "\n",
    "[Google Colab](https://colab.research.google.com/), by default, has Featuretools `0.4.1` installed. You may run into issues following our newest guides, or latest documentation while using an older version of Featuretools. Therefore, we suggest you upgrade to the latest featuretools version by doing the following in your notebook in Google Colab:\n",
    "```shell\n",
    "!pip install -U featuretools\n",
    "```\n",
    "\n",
    "You may need to Restart the runtime by doing **Runtime** -> **Restart Runtime**.\n",
    "You can check latest Featuretools version by doing following:\n",
    "```python\n",
    "import featuretools as ft\n",
    "print(ft.__version__)\n",
    "```\n",
    "You should see a version greater than `0.4.1`"
   ]
  }
 ],
 "metadata": {
  "file_extension": ".py",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "mimetype": "text/x-python",
  "name": "python",
  "npconvert_exporter": "python",
  "pygments_lexer": "ipython3",
  "version": 3
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
